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ABSTRACT 

Making  definitive  attributions  concerning  satellite  anomalies  proves  to  be  a  challenging  endeavor  given  the  dynamic 
space  environment,  the  threat  of  adversarial  actions,  and  unanticipated  system  failures.  Further,  decision  makers  are 
usually  contending  with  performance  shaping  factors  such  as  time  pressure  and  the  knowledge  that  errors  can  be 
extremely  costly.  Significant  consequences  can  emerge  with  erroneous  conclusions,  whether  it’s  failing  to  thwart  an 
adversary’s  attack  against  our  space  assets,  or  misconstruing  an  environment  effect  for  a  hostile  action  that  drives  a 
response.  Although  accurate  and  reliable  measurements  of  the  disturbances  in  the  earth’s  magnetic  field  and  the  flux 
of  high  energy  protons  and  electrons  have  been  available  for  decades,  it  remains  challenging  to  translate  these  data 
into  actionable  information  concerning  the  potential  threat  to  on-orbit  assets.  Even  though  satellite  operators 
actively  monitor  these  hazards,  until  very  recently  there  has  been  limited  statistical  relationship  between  the 
measured  radiation  environment  and  the  likelihood  of  an  anomaly  to  the  on-orbit  asset  [5],  To  address  this  need,  the 
Air  Force  Research  Laboratory  (AFRL)  is  supporting  work  to  make  space  environmental  effects  information  more 
accessible  and  actionable  for  users  in  the  operational  community.  The  tool  under  development  leverages  O’Brien’s 
“hazard  quotients”  (which  are  derived  from  historical  records  of  on-orbit  anomalies)  to  relay  the  potential  effect  by 
presenting  anomaly  likelihood  information  related  to  surface  charging,  internal  charging,  single  event  effects,  and 
the  total  accumulated  particle  dose. 


1.  Introduction 

Making  definitive  attributions  concerning  satellite  anomalies  proves  to  be  a  challenging  endeavor  as  analysts  are 
confronted  with  the  dynamic  space  environment,  the  threat  of  adversarial  actions,  and  unanticipated  system  failures. 
Accurate  assessment  of  space  weather  effects  is  a  valuable  capability  that  can  help  guide  the  attribution  process. 
Other  researchers,  including  Reeves  [1]  at  AMOS  2010,  advocate  the  need  for  a  rapid  assessment  tool  that  can 
provide  an  estimate  of  space  weather’s  role  in  an  anomaly.  Flowever,  the  physics  behind  space  weather  phenomena 
is  extremely  complex  and  requires  extensive  training  to  master.  The  discipline  has  tended  to  produce  reports  and 
findings  that  are  more  suited  for  expert  consumption  and  less  comprehensible  to  the  operational  community.  We 
therefore  are  striving  to  deliver  actionable  space  weather  information,  available  at  the  appropriate  level  of  detail  that 
supports  operationally-driven  workflows.  Taking  a  human-centered  approach,  Aptima  constructed  an  intuitive, 
interactive  interface  through  which  to  convey  space  weather  effects  and  support  data  exploration.  The  tool  is  crafted 
from  a  mission  perspective  and  is  intended  to  provide  a  first  order  assessment  pertaining  to  space  weather’s  potential 
for  causing  anomalies  at  specified  locations  and  times. 

2.  Understanding  User  Needs 

The  team  employed  components  of  the  Applied  Cognitive  Task  Analysis  (ACTA)  as  described  by  Militello  & 

Flutton  [2]  to  interact  with  space  weather  experts  and  operational  personnel.  Evidence  emerged  early  in  the  process 
suggesting  these  two  groups  did  not  speak  the  same  language  nor  share  similar  interests  on  many  levels.  More  than  a 
half  dozen  distinct  operational  user  groups  were  engaged  with  interests  ranging  from  satellite  operations  for  defense, 
intelligence,  and  scientific  study. 

The  ACTA  method  employs  three  tasks  to  engage  users  and  delve  into  the  critical  cognitive  factors.  The  first  is  a 
Task  Diagram  which  provides  a  broader  overview  of  the  task  but  drives  toward  the  more  difficult  cognitive  elements 
involved  in  the  task  and  results  in  decomposed  task  elements.  This  step  is  one  that  provides  improved  efficiency 
over  conventional  CTA  methods  as  the  information  gained  in  the  task  diagram  is  utilized  to  probe  more  directly  in 
the  next  two  steps  of  ACTA.  The  second  step  is  the  Knowledge  Audit.  The  purpose  of  the  knowledge  audit  is  to 
examine  how  expertise  is  utilized  in  the  domain  and  provides  concrete  examples  from  the  subject  matter  experts 
(SMEs)  direct  experience.  This  technique  draws  heavily  on  the  well-established  expertise  literature  (see  Chi  & 
Glazer  for  an  overview  [3]).  The  knowledge  audit  uses  a  set  of  probes  to  elicit  types  of  skills  and  domain  savvy 
central  to  the  task  as  well  as  encourage  the  interviewee  to  provide  relevant  examples. 
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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
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Task  diagram  and  simulation  interview  results  showed  divergence  between  the  user  groups,  indicating  a  bridging 
application  could  be  beneficial.  Thus,  this  investigation  was  bifurcated  to  gain  a  better  understanding  of  the  science 
behind  space  weather  phenomena,  and  the  information  required  for  operationally  relevant  decision  making.  Success 
would  depend  on  how  well  the  two  could  be  integrated  and  portrayed  to  operators. 

3.  Space  Weather  from  the  Operational  Perspective 

Despite  the  considerable  data  available  from  space-based  observations  and  ground-based  sensors  [4],  daily  space 
operations  often  fail  to  leverage  environmental  data  to  any  observable  benefit.  Investigation  reveals  two  primary 
barriers  to  integrating  space  environment  information.  First,  the  expertise  is  not  sufficient  at  the  operational  level  to 
assimilate  the  raw  environmental  data  for  mission  objectives.  While  there  are  numerous  environmental  measures 
scientists  can  produce  to  characterize  the  space  environment,  understanding  their  implications  for  mission-based 
tasking  proves  quite  challenging.  For  example,  conveying  particle  flux  measures  of  five  megaelectron  volt  (MeV) 
protons  does  not  embody  an  intuitive,  meaningful  quantity  relating  to  mission  impact.  Figure  1  provides  a  sample  of 
space  weather  data  that  are  available,  but  of  little  interest  at  the  operational  level. 
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Figure  1.  Space  environment  data  available  via  http://www.swpc.noaa.gov/today.html. 


The  second  obstacle  is  that  little  effort  has  been  put  forth  to  devise  a  better  way  of  presenting  space  weather 
information  to  users.  A  thorough  understanding  of  the  operators,  missions,  and  work  domain  is  needed  to  transform 
the  environmental  data  from  the  space  physics  observations  to  mission  relevant  inputs  for  the  operational 
community. 

Current  practices  at  the  operational  level  generally  include  distilling  all  the  available  data  down  to  a  universal 
assessment  that  space  is  green  (good),  yellow  (potentially  hazardous),  or  red  (disturbed)  for  a  given  period  of  time. 
The  time  may  vary  from  a  few  hours  to  a  24  hour  period.  Of  course,  global  assessments  also  assume  that  all 
locations  are  essentially  identical  and  offer  no  specificity  in  terms  of  asset  position.  With  a  space  environment 
characterization  so  stripped  of  information,  operators  conducting  particular  missions  with  geographic  constraints 
find  it  difficult  to  apply. 

The  user  investigations  we  performed  reveal  that  in  contrast  to  the  physicists,  the  operational  users  find  little  use  in 
receiving  particle  fluxes  or  magnetometer  readings  collected  by  the  scientific  community.  Fortunately,  another 
group  of  scientists  also  arrived  at  this  conclusion  and  have  developed  methods  for  quantifying  a  Hazard  Quotient 
relating  space  environment  readings  to  operationally  relevant  input.  Paul  O’Brien  [5]  at  the  Aerospace  Corporation 
devised  a  set  of  algorithms  for  the  Spacecraft  Environmental  Anomalies  Expert  System  for  geosynchronous  orbits 
(SEAES-GEO)  model.  Using  a  collection  of  historical  anomalous  events,  it  employs  recorded  data  from  the  space 
environment  to  determine  the  ratio  of  the  instantaneous  anomaly  likelihood  to  the  mission-averaged  anomaly 
likelihood. 


Using  these  models,  Aptima  constructed  an  application  that  collects  the  publicly  available  data  pertaining  to  particle 
fluxes  and  magnetic  indices  (http://www.swpc.noaa.gov')  to  compute  hazard  quotients.  Values  can  be  computed  for 
each  degree  of  longitude  (the  operators’  preferred  means  of  describing  position  in  geosynchronous  orbits).  The 
application  marshals  these  data  and  then  computes  hazard  quotients  for  four  space  environment  threats  (surface  and 
internal  charging,  single  event  upsets  and  total  dose)  which  are  displayed  in  the  familiar  red/yellow/green  stoplight 
fashion.  These  calculations  lend  themselves  to  a  more  operationally  relevant  translation  for  those  with  space 
situation  awareness  (SSA)  responsibilities. 

But  the  question  of  how  the  information  should  be  delivered  to  the  operator  remains.  The  lines  of  data  in  Figure  2 
are  uninspiring  and  largely  a  presentation  of  excessive  noise  with  little  signal  from  an  operator’s  perspective.  We 
needed  to  probe  the  mental  models  of  the  space  analysts  and  better  understand  their  diagnostic  procedures  and 
workflows  to  make  a  useful  application. 


Fig  2,  Computations  for  the  Internal  Charging  Flazard  Quotient. 

More  than  a  half  dozen  operational  user  groups  were  engaged  with  interests  ranging  from  satellite  operations  for 
defense,  intelligence,  and  scientific  study.  Even  among  the  operator  groups  we  discovered  some  surprising  findings 
and  differences  between  subgroups  (for  example,  one  group  of  operators  almost  never  relies  on  geospatial 
information).  Despite  some  divergence  in  the  details  of  their  needs,  several  opportunities  emerged  to  improve  the 
information  delivered  at  the  operational  level  that  would  cut  across  specialties. 


4.  Prototype  Development 

Operational  users’  frustration  with  current  space  weather  information  available  was  almost  universal  within  our 
sample,  resulting  in  a  general  dismissal  of  space  environment  data.  The  information  they  receive  to  inquiries 
characterizing  historical  space  weather  (as  well  as  the  daily  briefing  of  current  condition),  as  a  global  assessment 
undermined  the  reliability  and  usefulness  of  the  reporting.  Their  experience  as  operators  is  such  that  anomalies 
occur  on  what  are  considered  “good”  days  (i.e.,  little  in  the  environment  that  should  lead  to  anomaly)  as  well  as 
“danger”  days  in  which  they  saw  no  environmental  effects  at  all. 

One  of  the  more  robust  findings  among  the  operators  was  their  belief  that  more  defined,  reliable  space  weather 
information  could  be  valuable  in  their  forensic  investigation  of  known  anomalies.  That  is,  after  an  event  has 
occurred  (e.g.,  satellite  X  suffered  degraded  capabilities  yesterday  at  1700  UTC),  it  would  be  useful  to  know  the 


environmental  conditions  around  the  event  time  to  help  make  a  preliminary  assessment  of  the  space  weather’s  role 
in  the  anomaly. 

With  the  forensic  application  as  our  first  use  case,  the  design  specifically  sought  to  make  space  weather  effects 
information  clear  and  accessible  for  this  purpose.  Using  multiple  visualization  options,  the  user  will  be  able  to  find 
time  periods  of  interest  and  select  an  asset  location(s)  at  GEO  to  create  an  envelope  around  the  believed/known 
event.  Each  hazard  quotient  relates  to  one  of  four  satellite  threats  corresponding  to  the  SEAES-GEO  model:  surface 
charging,  internal  charging,  single  event  upsets,  and  total  dose.  Discussions  with  operational  personnel  provided 
assurance  that  these  concepts  have  meaning  for  them  in  terms  of  the  threat  to  the  asset  and  relevant  mission-level 
effects  that  may  be  of  concern.  Figure  3  depicts  the  Space  Awareness  Toolkit  for  Calculating  Anomalies  to  Satellite 
Tasking  (SATCAST)  application  set  to  view  four  different  satellites  (each  is  represented  by  a  vertical  set  of  dots 
corresponding  to  the  satellites  longitude).  Users  can  select  the  time  of  interest  and  the  longitude  (which  corresponds 
to  the  asset  under  investigation  in  GEO),  and  may  also  create  and  save  groups  (of  longitudes)  that  correspond  to  a 
constellation  of  interest  (as  is  shown  in  Figure  3) 


Fig  3.  SATCAST  showing  four  satellites  on  a  vertical  timeline. 


The  vertical  timeline  was  a  critical  feature  (opposed  to  the  more  widely  used  convention  in  the  western  world  which 
is  a  horizontal  timeline  in  which  the  past  is  to  the  left)  to  preserve  the  proper  location  of  a  satellite  at  a  given 
longitude.  With  a  vertical  timeline,  the  selection  of  0  longitude  remains  in  place  as  one  scrolls  up/down  to  go 
forward  or  backward  in  time.  Further,  it  retains  the  directional  mapping  of  the  scrolling  motion  to  the  change  in 
values.  A  horizontal  timeline  (with  a  slider  for  example  while  keeping  the  map  still)  would  require  the  user  to  make 
an  orthogonal  mental  rotation  which  is  likely  to  induce  errors  [6]  because  the  mapping  is  not  as  natural. 

Two  windows  are  open  that  contain  detail  on  the  time/location  of  the  asset  the  user  selected  (clicking  a  dot  opens  the 
window).  The  window  contains  the  longitude  and  time  information,  local  and  UTC  time,  and  the  high  level  indicator 
(red/yellow/green)  for  each  hazard  quotient.  This  window  provides  a  quick  look  at  the  potential  threats  at  a  glance 
for  each  of  the  four  hazard  quotients  for  the  time  frame  selected.  The  value  is  that  it  shows  the  operator  which 
hazard  quotient  is  driving  the  top-level  assessment  of  red/yellow/green.  In  Figure  3,  all  dots  are  red,  and  until  a 
window  is  opened,  it  is  not  clear  which  type  of  hazard  is  exceeding  that  threshold.  By  opening  two  windows,  both 
show  that  total  dose  is  the  dangerous  hazard.  Selecting  dots  that  are  separated  by  location  (right  to  left  in  GEO 
longitude)  and  time  (top  to  bottom  separates  them  in  hours),  it  is  seen  that  Total  Dose  is  in  the  red  and  single  event 


upset  is  yellow.  The  consistent  pattern  across  time  and  location  may  suggest  a  robust  environmental  effect.  These 
items  may  warrant  further  investigation. 

From  within  the  first  drill-down  windows,  an  operator  can  click  directly  on  one  of  the  hazard  quotients  to  open  a 
more  detailed  look  at  what  is  driving  the  value  of  that  hazard  quotient.  Figure  4  below  shows  drill-down  windows 
for  Single  Event  Upset  and  Total  Dose  hazard  quotients. 


Now  instead  of  a  simultaneous,  averaged  value  for  a  one-hour  block,  the  user  can  view  the  data  trending  over  a  48- 
hour  period  and  get  a  sense  of  hazard  fluctuations  over  time.  Note  again,  that  it  is  not  just  numbers  and  values  given 
to  the  operator.  A  clear  threshold  line  is  indicated  in  the  graph.  Individual  data  points  describing  the  hazard  (and 
depicted  in  their  appropriate  colors)  are  plotted  appropriately  to  indicate  the  level  of  concern  over  time.  Operators 
are  thus  unburdened  from  needing  to  know  or  memorize  a  “goodness”  range  of  values.  The  application  offers  a 
perceptual  understanding  that  is  consistent  with  many  other  systems  employed  at  the  operational  level. 

In  Figure  5  we  show  another  situation  in  which  a  single  asset  is  being  investigated.  The  first  level  window  is  seen  in 
the  middle  of  the  screen,  and  by  clicking  the  hazard  quotients,  it  has  opened  a  drill-down  window  for  each.  Starting 
in  the  upper  left  and  moving  clockwise,  more  detailed  information  is  shown  for  Surface  Charging,  Internal 
Charging,  Single  Event  Upsets  and  Total  Dose.  The  use  of  color  is  intended  to  facilitate  the  interpretation  of 
potential  space  environment  hazard  at  a  glance.  The  size  of  the  oval  is  also  determined  by  the  hazard  index  with  red 
producing  the  largest  object,  yellow  smaller,  and  green  smaller  still.  The  reduced  size  is  less  salient  and  is 
appropriate  because  of  its  reduced  need  to  call  the  operator’s  attention. 
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Fig  5.  SATCAST  with  drill-downs  shown  for  (clockwise  from  top  left)  Total  Dose,  Surface  Charging,  Internal 

Charging,  and  Single  Event  Upsets. 


5.  User  Influence 

Any  tool  with  low  reliability,  clumsy  interactions,  and  latent  utility  faces  tremendous  resistance  to  adoption.  To 
prevent  such  labels  from  plaguing  this  design,  we  have  engaged  the  operational  community,  for  whom  this  is 
intended  to  support,  to  understand  their  needs,  decision  processes,  and  work  context.  With  these  data,  we  needed  to 
take  great  care  in  not  alienating  the  user  population  we  were  determined  to  serve.  Again,  they  are  not  space 
physicists.  Though  a  few  conceded  some  of  this  detail  was  beginning  to  approach  their  limit  of  space  weather 
understanding,  most  assured  us  that  the  hazards  identified  had  meaning  for  their  space  operations.  The  drill-downs 
were  developed  with  and  vetted  by  several  operators  stating  this  level  of  detail  was  still  of  use  to  them. 

The  value  of  user  participation  cannot  be  overemphasized.  A  specific  design  element  pertaining  to  the  top-level 
display  serves  to  illustrate  their  value.  From  the  point  of  the  four  hazard  quotients,  we  received  strong  input  that  a 
roll-up  or  aggregate  would  be  needed  in  the  final  design.  Speaking  with  Paul  O’Brien  and  others  indicated  there  was 
no  easy  way  to  craft  a  mathematical  function  that  could  bring  together  all  four  quotients.  Their  properties  were  not 
conducive  to  aggregating  with  a  simple  function. 

In  contrast  to  the  complex  modeling  driving  this  element  of  design  in  our  approach,  the  benefit  of  working  with  the 
operators  again  emerged  with  their  parsimonious  offering.  After  engaging  several  mathematicians  and  space 
physicists  to  devise  valid  computational  formulas  for  aggregating  the  four  hazard  quotients  to  a  meaningftil  roll-up, 
the  operators  assured  the  best  approach  for  them  would  be  to  always  show  the  worst  case  of  the  four  hazard 
quotients.  This  would  of  course  provide  an  alert  when  at  least  one  parameter  was  out  of  normal  without  any 
averaging  effects  that  might  yield  a  failed  detection.  Further  investigation  will  determine  if  such  a  method  results  in 
too  many  false  alerts.  Nonetheless,  this  will  likely  be  one  of  the  display  modes  (and  perhaps  the  default)  on  their 
home  screen. 

Another  intriguing  design  challenge  the  team  is  devoting  significant  resources  to  is  determining  the  thresholds  for 
indicating  green-yellow-red  status.  These  thresholds  are  required  for  each  individual  hazard  quotient  as  well  as  the 
summation  status.  As  Figure  3  depicts  the  four  hazard  quotients  in  a  single  artifact  representing  the  aggregation  of 


space  weather  hazards  (the  single  bar  in  the  upper  right),  in  order  for  it  to  be  reliable,  must  be  calculated  with  a 
rigorous  and  robust  function.  Given  the  individual  hazard  quotients  are  derived  from  different  phenomena,  affect 
satellites  differently,  and  have  varying  mission  impacts,  a  simplified  weighting  approach  fails  to  provide  a 
meaningful  result.  This  stems  from  each  quotient  having  differing  and  non-Gaussian  distributions  (these  are  left 
bounded  at  0)  making  them  difficult  to  combine  along  a  common  scale. 

In  the  current  iteration,  we  are  taking  design  inputs  from  the  operators  pertaining  to  particular  interactions  they 
requested.  This  includes  functions  such  as  specific  viewing  settings,  selecting  temporal  bounds,  and  tracking 
spacecraft.  By  providing  a  flexible  platform  to  provide  information  at  the  appropriate  level  for  operator  expertise, 
work  domain  context,  and  mission  needs,  we  believe  we  can  provide  an  enhanced  capability  supporting  improved 
understanding  of  the  environmental  component  for  SSA. 

Work  with  space  weather  scientists  will  continue  to  verify  the  best  science  available,  but  more  importantly,  the 
iterative  design  processes  with  operational  users  will  shape  the  final  product.  It  is  hoped  this  application  can  become 
a  trusted  tool  for  operators  with  interests  in  managing  assets  in  geosynchronous  orbit,  as  that  is  where  the  science 
has  been  most  vetted.  Operators  at  other  orbits  have  also  expressed  interest  which  may  provide  paths  of  additional 
development. 


6.  Conclusion 

Environmental  factors  are  persistently  capable  of  affecting  a  satellite’s  performance  and  survivability  which 
ultimately  impact  U.S.  space  superiority  and  the  capabilities  delivered  to  the  warfighter.  These  space  assets  are  used 
daily  to  support  numerous  missions  by  relaying  communications,  to  ensuring  precision  targeting.  The  men  and 
women  in  the  military,  whose  expertise  is  crafted  for  mission  specialties,  are  unlikely  to  also  have  expertise  in  space 
physics.  We  therefore  are  striving  to  deliver  actionable  information,  available  at  the  appropriate  detail,  and 
conformant  to  their  workflow. 
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